Automatic Feature Extraction and Indexing for Content-Based Visual Query

Content-based indexing and query has been considered as a powerful technique for accessing large visual information systems (databases and video servers). By extracting and indexing the visual contents of the images such as texture, color, and shape, users may search desired images by specifying the image contents directly. However, a practical and economical solution cannot afford extensive user involvement. This paper proposes an approach in which image processing technology is explored to the limit of its capability for automatic extraction and indexing of image features. We relax the criterion of image content recognition that concrete 3D objects can be successfully segmented from 2D images. Instead, we aim to index the prominent image regions with distinctive features only. By combining multiple modalities of useful signal features, we hope to characterize the prominent image objects efficiently and effectively. In order to further explore the maximum synergy between feature extraction and other image processing tasks required in image databases, we also propose to extract the visual features directly from the compressed images. For existing large image archives, decoding of compressed image data is hence not necessary. For new image database design, merging feature extraction intelligence into the compression algorithm has recently been recognized as an important issue for image processing research. We use texture, shape, and video scene change detections as examples in describing this compresseddomain approach. All the proposed research are being incorporated into practical applications in Columbia University’s Multimedia testbed.